1. Technical Field
The present invention relates to traffic management within computer networks and more particularly, to volume reduction based on data analysis of the traffic.
2. Discussion of the Related Art
In a communication network, the rate at which data to be transferred over the network is generated often exceeds the capacity of the networking infrastructure. This is likely to occur in smart energy grids and other sensor-based systems which need to handle vast amounts of information produced by sensors. The problem also arises in inter-domain communication, when the capacity of the source domain is greater than that of the target domain (for example, traffic from an enterprise domain to a real-time domain or from a wired domain to a wireless.) On a smaller scale, two nodes that communicate via a low-bandwidth channel may run into a similar problem.
Currently there are two complementary approaches that may alleviate the problem by decreasing the transmission rate. One is delaying transmission of messages so that the resulting rate does not overwhelm the network. The other is rearranging/compressing the data into a more compact form, possibly discarding a portion of it. The two approaches are referred herein as traffic shaping and volume reduction, respectively. While the former addresses temporary load spikes, the latter has also the ability to resolve long-term bandwidth deficiency.
Among the most widely used traffic shaping methods are: (i) congestion control and (ii) flow control. Congestion control is the process of controlling traffic entry into a network, so as to avoid congestive collapse by attempting to avoid oversubscription of any of the processing or link capabilities of the intermediate nodes and networks. Flow control is the process of managing the rate of data transmission between two nodes to prevent the sender from overwhelming the receiver.
The volume reduction methods can be classified into lossless compression, message filtering and application-specific optimizations. Lossless compression employs a generic algorithm to compress data on the transmitter end and to decompress it to its original form on the receiver end. Message filtering policies, such as DDS HISTORY QoS and TIME_BASED_FILTER QoS allow discarding outdated or superfluous messages. Application-specific optimizations are custom methods that allow dropping entire messages, discarding the less critical parts of a message converting messages into a more compact format and aggregating a number of messages into one.
The aforementioned techniques may alleviate the problem to a certain degree, but none can solve it entirely in a sufficiently wide range of scenarios and applications. The traffic shaping methods may allow preventing network collapse in the presence of short bursts of data generation, but they are ineffective when the average data generation rate exceeds network capacity. Lossless compression may not compress the data sufficiently despite consuming a considerable amount of CPU. In addition, existing message filtering policies do not give an adequate solution for a large variety of applications, while application-specific optimization methods must be devised anew for each application.
In addition, a major drawback of existing volume reduction techniques is that they do not take into account the currently available amount of bandwidth, which is likely to change dynamically during the system operation. Their functionality is orthogonal to the state of the networking infrastructure. Consequently, reduction methods such as message filtering and application-specific optimizations often tend to discard more data than needed in order to meet the current network load.